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REVISITING THE FORWARD EQUATIONS FOR 
INHOMOGENEOUS SEMI-MARKOV PROCESSES 

ALEXANDER SOKOL 


Abstract. In this paper, we consider a class of inhomogeneous semi-Markov 
processes directly based on intensity processes for marked point processes. We 
show that this class satisfies the semi-Markov properties defined elsewhere 
in the literature. We use the marked point process setting to derive strong 
upper bounds on various probabilities for semi-Markov processes. Using these 
bounds, we rigorously prove for the case of countably infinite state space that 
the transition intensities are right-derivatives of the transition probabilities, 
and we prove for the case of finite state space that the transition probabilities 
satisfy the forward equations, requiring only right-continuity of the transition 
intensities in the time and duration arguments and a boundedness condition. 
We also show relationships between several classes of semi-Markov processes 
considered in the literature, and we prove an integral representation for the 
left derivatives of the transition probabilities in the duration parameter. 


1. Introduction 


Since the introduction of semi-Markov processes in [18] and [24], this class of 
stochastic processes have been thoroughly developed and applied in many fields 
of study. Initially, the semi-Markov processes studied were homogeneous semi- 
Markov processes, see e.g. [laiisiiiiii]. Intuitively, these are similar to homoge¬ 
neous Markov processes, except that the intensities of jumps depend on the amount 
of time spent by the process in its current state. This duration dependency is at 
the center of semi-Markov theory. 

Inhomogeneous semi-Markov processes, where the intensity for jumps depend on 
both time and the time spent by the process in its current state, have been studied 
as well, see e.g. [IQIIIIIEI]. Here, the theory becomes more complex, but also 
allows for more flexible modeling. 
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The strength of semi-Markov theory is its ability to model systems where the 
change of the state depends on the amount spent in its current state. Many 
real-world systems exhibit this type of behaviour. One obvious example is in the 
context of disability insurance, where the intensity for recovery depends on the 
amount of time spent in a disabled state: Intuitively, if one does not recover within 
a few years, it is considerably less likely that one will ever recover. Applications of 
semi-Markov theory to disability insurance, and life insurance in general, can be 
found in e.g. [21 [mil]. Semi-Markov models find applications elsewhere as well, 
in fields as diverse as for example wind speed modeling, high frequency finance, 
tourism movements and credit risk, see [6l[5l[l7l[T9]. 

The theoretical literature on semi-Markov processes is considerable, covering e.g. 
[221 [a m [ini E EQi E] and much more, see the books [mile] for thorough refer¬ 
ence lists. The majority of the literature, however, focuses on the homogeneous 
case, neglecting the inhomogeneous case. Furthermore, sometimes the interest in 
applications take precedence to rigor and mathematical precision, with e.g. [10] 
stating: We shall give a few proofs, but not where the results can be argued by 
’’direct reasoning” (i. e., by intuition) ... All this means that we shall sweep some 
interesting mathematical problems under the rug. 

In this paper, we revisit some fundamental results for semi-Markov processes, 
namely the definition of semi-Markov processes, related Markov properties, and 
the characterization of intensities and transition probabilities. In particular, we 
give precise conditions and a rigorous proof for the transition probabilities to 
satisfy the forward partial integro-differential equations, utilizing a direct proof 
based on estimates on the probabilities of multiple jumps in a short time interval. 
The forward equations is one example of a result related to the citation by nni 
above: It is intuitively quite clear that the equations should hold under ’’sufficient 
regularity conditions”, but obtaining a rigorous proof of this is challenging. 

All of our results cover the inhomogeneous case. Furthermore, we also consider the 
case of countably infinite state spaces when possible. Our results not only infuse 
known results with new and improved proof methodologies, but also generalize 
known results. Specifically, our contributions are as follows: 


(1) . We prove a theorem on the relative sizes of various classes of semi-Markov 

processes considered in the literature. 

(2) . We prove an upper bound for the conditional probability of a semi-Markov 

process making two jumps in a small interval, Lemma l3.ll and demonstrate 
in our proofs that this lemma can be used as a key tool for rigorous proofs 
of various analytical properties of semi-Markov processes. 

(3) . In the case of a countable state space, we give a precise sufficient condition 

on the intensities of a semi-Markov process for being the derivatives of the 
transition probabilities. 
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(4). In the case of a finite state space, we give a precise sufficient condition on 
the intensities of a semi-Markov process for satisfying the forward partial 
integro-differential equations. 

The remainder of the paper is organized as follows. In Section [21 we review the 
various notions of semi-Markov processes considered in the literature, we introduce 
a class of semi-Markov processes with extra regularity, and we prove a theorem 
on the relative sizes of various classes of semi-Markov processes. In Section |3l we 
prove a bound on the probability of a semi-Markov process making two jumps in a 
short interval of time, and we give sufficient conditions for the transition intensities 
to be the right derivatives of the transition probabilities. In Section (H we give 
sufficient conditions for the transition intensities to satisfy the forward equations. 
Finally, in Section |5l we discuss our results and consider opportunities for further 
research. Proofs can be found in Appendix [^ 


2. Weak and strong classes of semi-Markov processes 


Several definitions of the notion of a semi-Markov process exist in the literature, 
varying in their strictness. In this section, we review the definitions used in the 
literature, and we introduce a class of semi-Markov processes with extra regularity, 
using intensity processes for marked point processes. Furthermore, we prove a 
result on the relative sizes of various types of semi-Markov process. 

We work in the context of a sequence of random variables (Yn, Sn)n>o, where 
5'o = 0 and 5'„ > 0 for all n > 1 and Yn takes its values in E for all n > 0, where 
{E,£) is some measurable space. We assume that l^+i ^ Yn for all n > 0. We 
then also define T„ = for n > 0. In the context of a marked point process, 

see [12j . we think of (T„)„>i as the event times and of Yn as the marks, with Tq 
being time zero and Yq being the initial mark. We then also define Zt = Yn for 
Tn <t < Tn+i, and define Ut = t — sup{0 < s < t\Zs ^ Zt}. We refer to U as the 
duration process of Z. Intuitively, Ut measures the amount of time spent by Zt in 
its current state. Here, it is implicit that if Tn+i = oo for some n, then Zt = Yn 
for all t > Tn- Note that using this definition, it is immediate that Z has cadlag 
sample paths. 

One formulation of the semi-Markov property often seen is given directly through 
the dependence structure of (Yn, Sn)n>o- In [ 12 ] j the process Z is defined to be a 
semi-Markov process when 

(2.1) P(Yn=J,Sn<t\Yo,So,Yi,Si,...,Sn-l,Yn-l)=PY„_t,j(t) 

for some family P{i,-) of distributions on E, where Pij{t) = P{i,{j} x [0,t]). 
Here, then, Pij(t) can be interpreted as the probability of transitioning from i 
to j, having stayed in the previous state for a duration of less than or equal to 
t. This definition corresponds to a time-homogeneous case, and essentially states 
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that {Yn,Sn) is conditionally independent of So,Yo,. ■., Sn-i given Yn-i- The 
same definition is also used in [4], where the family {P{i,-))i^E is referred to a 
semi-Markovian kernel. The papers 0 US] also use variants of this definition. 
Furthermore, [9] defines Z to be a semi-Markov process if (T„, T„)„>o is a Markov 
process. The majority of authors take the case of finite state space E as their 
main interest. Some exceptions to this are [a EH El], who consider the case of a 
countable state space, and [26], who allows a general, possibly uncountable state 
space. 

The other main type of definition considers the process {Z,U). In particular, in 
[27], Z is said to be a semi-Markov process when {Z, U) is a strong homogeneous 
Markov process. This type of definition is also applied in OE], allowing for 
time inhomogeneity. In |10] the author endeavors to retain the discussion on an 
intuitive level, but generally also argues for a definition of semi-Markov processes 
of the same type. 

The definition based on having {Z, U) be a Markov process has several qualities: 
It encapsulates the central idea of a semi-Markov process, namely dependence on 
only the current state and the duration spent in that state, it is concise and it 
is equally coherent for both finite, countable and uncountable state spaces. As 
stated in the following definition, we will consider this to be the main defining 
property of a semi-Markov process. For sake of tractability, we require that [Z, U) 
in fact be a strong Markov process. 

Definition 2.1. Let {E,£) be a measurable space, and let Z he a cadlag process 
taking its values in E. We say that Z is an inhomogeneous semi-Markov process 
if {Z, U) is an inhomogeneous strong Markov process on E x R+. 

By itself, however, processes such as those given in Definition 12.11 are not suffi¬ 
ciently regular to allow for a rich mathematical theory. We next introduce a more 
regular type of semi-Markov processes, based on intensity processes for marked 
point processes. Our claim is that this class of processes is amenable to discuss 
various types of regularity and will provide a sound framework for rigorous devel¬ 
opment of theory. 

In the following, we assume given for each i,j G E with i ^ j a measurable 
mapping qij : —>• ]R-|_. We also define qi : K+ for i e A by 

(2.2) q^{t,u) = '^q,j{t,u), 

and we assume throughout that 

(2.3) sup swp qi{t,u) < oo 

{t,u)^K i^E 

for all compact subsets K of This will in particular ensure the absence of 
explosion for all processes under consideration. The requirement (lOl) may appear 
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strict at first sight, but really states little more than that all intensities for making 
jumps are bounded simultaneously on finite intervals of time. In the case where E 
is finite, the supremum over E is of course of no impact. We furthermore define 
qii{t, u) = —qi{t, u), and we let Q{t, u) denote the E x E matrix whose entry for 
the i’th row and j’th column is qij(t,u). 

Definition 2.2. We say that a marked point process process Z with countable 
state space E is an inhomogeneous semi-Markov process with intensities if there 
exists a family of mappings qtj : —>■ IR.+ for i ^ j satisfying (12.311 such that Z 

has intensity process A given by \t(i) = Ut-) for i € E, where we 

assume that each qi is Lebesgue integrable on compact subsets of 

Note that by Corollary 4.4.4 of [12], the bound (|2.3|) ensures that explosion does 
not occur and thus Definition l2.2l is not vacuous. In Theorem l2.31 we show that the 
processes defined in Definition 12.21 are in fact semi-Markov processes in the sense 
of Definition [O The requirement that qi is Lebesgue integrable on compact 
subsets of is made to ensure that the resulting intensity process {t,i) i-5> Xt{i) 
in fact is integrable and thus allows for a corresponding compensator process. The 
interpretation of Definition 12.21 is straightforward: When the process Z has been 
in state i for a duration u at time t, the intensity for making a jump to a state 
j is qij{t,u). 

We now show a result, Theorem l2.41 about the relative strengths of various notions 
of semi-Markov processes. The result that Definition 12.21 implies Definition 12.11 is 
sufficiently important to us to be stated as a separate theorem. 

Theorem 2.3. A ssume that Z is an inhomogeneous semi-Markov process with 
intensities. Then {Z,U) is an inhomogeneous strong Markov process on E x R+. 

Theorem 2.4. Let Z be a cadlag stochastic process with countable state space E, 
let (Tn) be the jump times of Z, and let U be the duration process of Z. Consider 
the following four statements: 


(1) . The process Z is an inhomogeneous semi-Markov process with intensities. 

(2) . The process Z is an inhomogeneous semi-Markov process. 

(3) . {Yn, Sn) is conditionally independent of Sq, Yq, ..., Sn-i given T„_i. 

(4) . {Yn,Tn)n>o « discrcte-time Markov chain. 


Here, (1) refers to Definition 12.A and (2) refers to definition Dehnition \2. 1\ It 
then holds that (1) implies (2), (2) implies (3), and (3) implies (4). 


In the next section, we consider results characterizing the distribution of a semi- 
Markov process with intensities, in particular characterizing the transition proba¬ 
bilities of the process. 
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3. Intensities as derivatives of transition probabilities 


Our objective in this section will be to rigorously obtain sufficient regularity crite¬ 
ria for the intensity functions Qij to ensure that they can be obtained as the right 
derivatives of the transition probabilities of the semi-Markov process. Further¬ 
more, we will show that under the same regularity conditions, the convergence of 
the difference quotients occur under the presence of a dominating bound. This will 
be essential for our later proof that the transition probabilities satisfy the forward 
equations. 

In the following, let Z he a, semi-Markov process with intensities qij for i ^ j. As in 
the previous section, we assume that the intensities satisfy (lOl) . As the process 
{Z, U) is a Markov process by Theorem 12.31 we may associate to it a family of 
transition probabilities Ps^t{{hu)^ ■) on E x IR+ such that 

(3.1) P{{Zt, Ut)&AxB\Ps) = PsAZs, Us, A X B), 

where ACE and B C B+, with B+ denoting the Borel cr-algebra on R+. Consis¬ 
tently with the notation outlined in [10] and also used in [2], we define, ioi i,j G E 
and s,t,u,d > 0 with s <t, 

(3.2) pij(s,t,u,A) = PsAhu,{j} X A), 

(3.3) pij(s,t,u,d) =pij(s,t,u, [0,d]), 

(3.4) pij{s,t,u) =pij(s,t,u,R+), 

where A C B+. We then think of Pij(s, t, u, A) as the probability of transitioning 
from i to j from time s to time t when the duration at time s is m, and requiring 
that the duration at time t is in A. We then also similarly think of pij(s,t,u) 
as the probability of transitioning from i to j from time s to time t when the 
duration at time s is u. Finally, as a function of d, Pij(s, t, u, d) is the cumulative 
mass function for the measure A i—>■ Pij(s, t, u, A). 

Our objective is to show that the intensities Qij are right derivatives of transition 
probabilities. To this end, we first state two lemmas of independent interest to be 
used in the proof. For use in the following, we define 

(3-5) = X! ^(Zs-^z,), 

0<s<i 

the counting process for counting the jumps of Z. Also, when Q is some E x E 
matrix, we let ||Q||oo denote the supremum norm of the matrix. Note that even 
when E is infinite, it always holds for t, m > 0 that 

(3.6) sup ||(5(t,u)||oo = sup sup |gij(t, it)| = sup snpqi{t,u) < oo, 

{t,u)^K {t,u)^K i,j^E {t,u)GK i^E 

for compact subsets K of due to the condition p.3p . Finally, we say that the 
intensities (qij) are right-continuous if it holds for all i ^ j that qij(t + h,u + k) 
tends to qij(t, u) whenever h and k tends to zero from above. 
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Lemma 3.1. It holds that 


P{Nt+h-Nt>2\Zt=i,Ut=u) 


(3.7) 




< 


||(5(s,m + s - i)||oo||(5(ii,'y 


s)||oodz; ds. 


Lemma 3.2. It holds that 

(3.8) lim lp{Nt+h -Nt> 2\Zt =i,Ut = u) = 0. 

h^O fl 


Lemma [3^ essentially shows that for semi-Markov processes satisfying the bound 
(EH), the probability of having two jumps in a small time interval tends to zero 
at a faster than linear rate. This is an important regularity property which is one 
of the key properties allowing us to prove our results on semi-Markov processes. 
The lemma follows immediately from Lemma l3. II Lemma l3. II is a stronger bound, 
allowing for a variety of derived bounds on probabilities for semi-Markov processes. 

We are now ready to state our two main results of this section. Theorem [T4] shows 
that the intensities are limits of transition probabilities, in accordance with our 
intuitive understanding of intensities. Theorem 13.31 shows that the convergence in 
Theorem 13.41 occurs under the presence of a dominating bound. This latter result 
is used in the proof of Theorem l3.4l and will also be essential for our rigorous proof 
of the forward equations in the next section. In the statement of the theorems, we 
let I denote the E x E identity matrix. 

Theorem 3.3. If the intensities are right-continuous, there exists a measurable 
mapping C : —>• IR.+ , bounded on compacts, such that there is e > 0 with the 

property that for any t,u > 0 and 0 < h < e, we have 



Theorem 3.4. If the intensities are right-continuous, it holds that 
(3.10) lim —(Pit, t -\- h,u) — I) = Qit, u). 

h^O h 


4. The forward equations 


Next, we turn to the characterization of transition probabilities. For homogeneous 
Markov processes, a closed-form expression for the transition probabilities is avail¬ 
able to us through the matrix exponential, see e.g. Section 2.8 of EH, and for 
inhomogeneous Markov processes, the transition probabilities can be character¬ 
ized through two different systems of multidimensional ODEs, namely the forward 
and backward equations, as proven in [8]. In the semi-Markov case, the transi¬ 
tion probabilities also satisfy forward and backward systems of equations. In the 
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case of the forward equations, these take the form of a system of partial integro- 
differential equations. The existence of these equations for semi-Markov processes 
is well known. That the transition probabilities satisfy both the forward and 
backward equations is proved very rigorously in for the case of a homogeneous 
semi-Markov process. A variant of the forward equations for the inhomogeneous 
case is stated in [2] , where it forms the basis for numerical calculations of cashflows 
in life insurance. 

In this section, we use the results from Section [3] to give a direct and rigorous 
proof of a sufficient condition on the intensities for the transition probabilities to 
satisfy the forward equations. We will also show that this set of equations can be 
rewritten as a set of ordinary integro-differential equations in two different ways. 

Throughout this section, we assume that E is finite and that the transition proba¬ 
bilities are right-continuous. The condition (12. also remains in force, but is now 
equivalent to the simpler condition 

(4.1) sup qi{t,u)<oo. 

{t,u)GK 

Before proving that the transition probabilities satisfy the forward equations, we 
require two lemmas of independent interest. The first lemma states that in an as¬ 
ymptotic sense, no double jumps back and forth are made over a small time period 
by a semi-Markov chain. Here, we define Pij{s, t, u, d—) = lim^_,.o+ Pij (s, t, u, d—h) 
for d> 0. 

Lemma 4.1. It holds that 

(4.2) lim -rPiiU, t -\- h,u, (u -\- h)—) = 0. 
h^o h 

Furthermore, there exists a measurable mapping C : —>■ M+, bounded on com¬ 

pacts, such that there is e > 0 with the property that for any t,u > 0 and 0 < h < s, 
we have 

(4.3) t + h,u,{u + h)-) < C{t, u). 
h 

Note that the result in Lemma |4T] cannot be extended to cover pii{t,t-\-h, u, u-\-h) 
instead oipii{t,t-\-h,u, {u-\-h)—), since pii{t,t-\-h,u, ■) is concentrated on [0,u-|-/i] 
and so it holds that 

lim -rPifit, t-\-h,u,u-\-h) = lim --piiit, t -\-h,u) 

h —^0 /l h —^0 iL 

= lim Y{ptiit,t-\- h,u) - 1) + y, 
h h 

which is infinite, according to Theorem 13.41 Essentially, this shows that in an 
asymptotic sense, the only mass for pii{t,t -\- h,u,-) is a point mass 'm u-\- h, 
corresponding to no jumps being made when transitioning from i to z from time t 
to time t -\- h. 
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We are now ready to begin work on proving the differential properties of the 
transition probabilities. We begin by showing that the transition probabilities 
are differentiable from the left in the final parameter, and we derive an explicit 
expression for the derivative. With this lemma in hand, we will be able to prove 
our main result on the forward equations. 

Lemma 4.2. Let i,jGE, let 0 < s < t and let u >0. It holds that the mapping 
d I—>■ Pij{s,t,u,d) is differentiable from the left for d > 0. Furthermore, for i j 
and d > t — s, the derivative is zero, for i j and d < s — t, the derivative is 



(4.4) 


where 


(4.5) akj{t,d,u,v) = Qkjit - d,v)exp J qj{r,r - {t - d)) dr j . 

The intuitive explanation for the formula (14.41) is as follows. The derivative is the 
limit as h tends to zero from above of 

i {pij{s,t,u,d) -pij{s,t,u,d- h)) = ^p^j{s,t,u, {d - h,d\) 



(4.6) 


Now, having d — h < Ut < d is equivalent to having a jump being made in the 
time interval [t — d,t — d h) and having no jumps made in the time interval 
[t — d + h,t]. Conditioning on Zt-d = k for k G E yields that the the differential 
quotient (14.6|) approximately is the sum over the states k, and for each state, we 
sum the density of transitioning from k to j immediately after time t — d and 
then remaining in state j from time t — d to time t with duration zero at time 
t — d. Furthermore, this is weighted with the probability of transitioning from 
state i to state k from time s to time t — d with duration u at time s. The term 
corresponding to k = j vanishes, as being in state j at time t — d would indicate an 
ultimate duration at time t greater than d. In (14.4L the integration with respect 
to Pik{s,t — d,u, dn) represents conditioning on the state at time t — d, the left 
factor in (14.5p corresponds to the conditional density of making a jump from k to 
j immediately after time t — d, and the right factor in (14.51) corresponds to the 
probability of remaining in state j from time t — d to time t with duration zero at 
time t — d. 


We are now ready to prove that the transition probabilities satisfy the forward 
partial integro-differential equations. 



10 


A. SOKOL 


Theorem 4.3. Fix s > 0 andu > 0. It holds fort > s and d > 0 that pij{s,t,u,d) 
is differentiable in t from the right, and the derivative is 



(4.7) 


where the partial derivative with respect to d is the derivative from the left. 

Theorem |T3] yields the semi-Markovian analogue of the Kolmogorov forward equa¬ 
tions for Markov processes. As an immediate corollary of Theorem 14.31 we may 
also derive a system of ordinary integro-differential equations for the transition 
probabilities. Applying the chain rule, it holds that 



and applying Theorem 14.31 in (14.81) . we obtain 




(4.9) 


Note that in (14.81) and (14.9p . the term 


(4.10) 


—Pij{s,t,u,dF t - s) 


refers to differentiation of the composite mapping pij{s,t,u,d + t — s), and not 
to the derivative of Pij{s, t, u, d) evaluated in (s, t,u,d + t — s). Also note that in 
the case of a Markov process, where qij{t,v) = qij{t) for some qij : R+ —>■ IR+, we 
may let u = d in (14.91) and immediately recover the classical Kolmogorov forward 
equation from [S]. 

Furthermore, we may also insert our expression (14.41) for the derivative in d of 
Pij{s,t,u,d) directly into (14.7p to obtain 




qkj{t,v)p^k{s,t,u, di;) -I- / qjj{t,v)pij{s,t,u, du) 



(4.11) 


for the case i ^ j and d < t — s, with a similar result holding for the diagonal case. 
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5. Discussion 


In this article, we have considered a class of regular semi-Markov processes de¬ 
fined through intensities for a marked point process. We have proven relationships 
between different notions of semi-Markov processes, and we have shown how to 
obtain sufficient conditions on the intensities for having the intensities be the right 
derivatives of the transition probabilities. Finally, we have given sufficient condi¬ 
tions on the intensities for having the transition probabilities satisfy the forward 
equations, and we have stated the latter equations in three different variants: in 
one way as a system of partial integro-differential equations, and in two ways as 
a system of ordinary integro-differential equations. In the course of this, we have 
also proved a formula for the derivative of the transition probabilities with respect 
to the duration d. 

Our main purpose and focus has been to develop a framework and tools for rig¬ 
orous analysis of semi-Markov processes. Our results indicate that analysis of 
semi-Markov processes centered around conditioning arguments for single jumps, 
combined with applications of bounds for probabilities of several jumps such as 
Lemma o and Lemma 13.21 is a fruitful proof methodology. 

We feel that there is ample room for improvement of our results and for continued 
study of semi-Markov processes. Some subjects for further study of semi-Markov 
processes which we find particularly interesting are as follows: 


(1) . Assuming sufficient regularity conditions, the transition probabilities also 

satisfy a different system of equations, the backward partial differential 
equations, see e.g. nna, and see [5] for a rigorous proof in the homoge¬ 
neous case. It is of interest to develop rigorous arguments for sufficient 
conditions on the intensities ensuring the validity of the backward equa¬ 
tions. 

(2) . In terms of the forward equations, it is of interest to weaken the regularity 

conditions required for the intensities, and to consider the case of countably 
infinite state spaces. In the latter scenario, several interchanges of limits 
and summation would have to be argued for separately. 

(3) . The forward and backward equations can be used for numerical evaluation 

of various expressions in semi-Markov models, see e.g. [2]. In terms of 
numerical studies, it would be of interest to compare the computational 
efficiency and accuracy of numerical solutions of these equation systems. 


We hope that our efforts in this paper will inspire further study of the field of 
semi-Markov processes. 
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Appendix A. Proofs 


A.l. Proofs for Section!^ 


Proof of Theorem \2.S[ We may assume without loss of generality that Z has 
deterministic initial state ?/o- We begin by proving that the process {Z,U) is 
a piecewise deterministic Markov process obtained from a marked point process 
satisfying the sufficient criteria of Theorem 7.3.1 of [12]. In order to do so, we 
need to specify both the underlying marked point process as well as the transform 
functions (^s,t appearing in that theorem. 


Define G = E x K_|_. Letting E be endowed with the discrete topology and 
letting K+ be endowed with its usual topology, we endow G with the product 
topology. With Q denoting the corresponding Borel cr-algebra on G, it holds that 
G is countably generated and contains all singletons, as is required by the results of 
Section 7.3 of [T2|. Next, let Zt = {Zt, 0) and define T„ = and Yn = {Yn, 0). We 
then obtain that Z is a marked point process with mark space (G, G). Furthermore, 
for 0 < s < t and with y = {y,u), define (j)s,t : G —>■ G by (j)s,t{y) = {y,t — s). Also 
define = supITtiITti < t} and define Y^t) = Yf^^^ ■ We then obtain 

(A.l) {Zt,Ut) = 

This shows that (Z, U) can be obtained as a piecewise deterministic process as in 
Theorem 7.3.1 of [12] . with underlying marked point process Z. It remains to check 
the requirements on Z of that theorem. In the notation of [12] . the distribution 
of the marked point process Z satisfies 

(A,2) = exp 1-%„(s,s-(„)dsV 

(A.3) x<;;,(((j,o)}) = 

QHn W ’ ^ ) 

for t > 0, n > 1 and j G E with (j, 0) ^ jjn and with (jA.3|) the latter being 
defined to be zero when the denominator is zero, the value of 0)}) in 

this case is irrelevant for the distribution of the marked point process. Here, 
Zn = (ti,yi,... ,tn,yn) and = (y„,0). Also, similar expressions are obtained 
for the distribution of (Ti,Fi). With y = {y,u) G G, now define qt{y) = qy{t,u). 
Noting that ipo^tiy^u) = {y,t), we obtain 

(A.4) qvois,s) = qs{yo,s) = qs{</)o,s{yo)), 

(A.5) qy„is,s- tn) = qs{yn,S- t„) = qs{4’U,s{yn)), 


SO that with t > 0, {y, u) G G and C G G and 


(A.6) 


rt{{y,u),G) 


U,'^)eC 


qy{t,u) ’ 
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we obtain 

(A.7) = exp qs{.4>t,,,s(.yn)) ds^ , 

(A.8) = rti(l>t„,tiyn),C). 

Finally, we have that 

qs{(l)s,t{y))^s = J qy{s,t - s)ds 

is finite for small h by assumption, so that (i) of Theorem 7.3.1 of [T^] is satisfied. 
As the remaining requirements (ii) and (iii) are trivially satisfied, we may invoke 
the theorem and obtain that {Z^ U) is a piecewise deterministic Markov process 
obtained from a marked point process satisfying the sufficient criteria of Theorem 
7.3.1 of [H]. Theorem 7.5.1 of [H] then furthermore shows that (Z, U) is a strong 
inhomogeneous Markov process. □ 

Lemma A.l. For all n > 1, it holds that when Tn < oo, we have Zt^ = Yn and 
Zt^- — ZTn_^. Furthermore, for Tn < t < Tn+i, it holds that Ut = t — Tn- In 
particular, Ut„- = Sn, with Sn = Tn — Tn-i- 


Proof of Lem,m,a \A.l[ The result on the values of Zt„ and Zt^-i follow immediately 
from the pathwise definition of Z in terms of r„ and Yn- As regards the claims 
about U, let Tn < t < Tn+i- It then holds that Z is constant on [T„,t], while it 
holds that = Yn-i f^Yn = Zt„, so Z changes its value at Tn- Therefore, we 
have 

(A.IO) sup{0 < s < t\Zs ^ Zt} = Tn, 

and so l/xn = t — Tn- In particular, it follows that 

(A.11) {7t„- = lim UT„-h = lim (r„ - h) - Tn-i =Tn- Tn-i = Sn, 

Ii->0+ /t—>-0+ 

as required. □ 

Proof of Theorem \2.4\ That (1) implies (2) is the content of Theorem l2.3l To prove 
that (2) implies (3), assume that (Z, U) is an inhomogeneous Markov process. Note 
that as So = 0, it suffices to prove that (Yn,Sn) is conditionally independent of 
Yq, Si , Yn- 2 , Sn-i given T„_i. Noting that we always have Ut„_i = 0, we may 
apply Lemma [A. II and the strong Markov property at the stopping time Tn-i, 
obtaining 

P{Yn — j, Sn Si f'\Yo — ioi Si = Si, . . . , Sn—1 — Sn—l, Yn — l = in—l) 

— — ji ^Tn — — f\Yo — f 0: ‘8'l — Si, . . . , Sn—1 — ^n—1, Yn—l — fn — 1) 

= P{Zt,, = j, Ut„- < t\UT„_i = = in-l) 

(A.12) =P{Yn=j,Sn<t\Yn-l=ln-l), 

which shows the desired conditional independence statement. As regards the proof 
that (3) implies (4), we note that with Sk = tk — tk-i for 1 < A: < n — 1, it holds 
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with a = t — tn-1 that 

PiYn = j, Tn < t|lb = *0) Ti = tl, . . . , Tn-l = F„_i = 

— -P(hn — J ; Sn ^ — ^0 5 5 ■ ■ • ; *^n—1 — -^n—1 : — Zt^—I ) 

(A.13) = p(y„ = j,Sn<t- t„_i|y„_i = z„_i). 

As this does not depend on ti,..., tn-2, and neither on lo,..., Zn-2, we conclude 
that {Yn,Tn) is conditionally independent of do, To, ■ ■ ■, Yn-2,Tn-2 given Yn-i and 
T„_i, yielding the desired discrete-time Markov property. □ 


A.2. Proofs for Section [3j 


Before proving the main results of Section [3l we first prove a series of lemmas. 
To this end, we fix some notation and recall some facts about hazard functions 
and hazard measures. First, for t > 0 and any stopping time S with respect 
to the filtration (Tt) induced by Z, let T{S) denote the first jump strictly after 
time S. Note that as we have T{S) = inf{T„|rji > S'} = infand 
(T„ > S) G see e.g. Section Lib of [13], it holds that T(S) is a stopping time 
for any S. Also, recall that for a distribution fi on ]R_|_ U { 00 } whose restriction to 
K+ has density / with respect to the Lebesgue measure, we may define the hazard 
function h : K_|_ ^ R+ of fj, by h{t) = 00 ]) whenever the denominator is 

nonzero, otherwise we let h(t) = 0, and it then holds that 

(A.14) ^((t, 00 ]) = exp y h{s)ds 

(A.15) f{t) = h{t) exp J h{s) ds 

For distributions which are not absolutely continuous, the more general construct 
of a hazard measure can be applied, see Section 4.1 of [12]. The following four 
lemmas yield distributions and hazard functions for various variables related to 
the distribution of Z. In the following, we let D{E) denote the set of cadlag 
mappings from ]R_|_ E, and generally apply the notation of |12| when considering 
distributional properties of marked point processes. 

Lemma A.2. The conditional distribution of Tn+i given ... ,Tn,Yn has 

hazard function h : ]R_|_ —>■ ]R_|_ given by h{t) = l(t>T„)9Y'„(i,^ ~ Tn). 


Proof of \A.S[ With Zn = {to,yo, ■ ■ ■ ,tn,yn), let denote the conditional distri¬ 
bution of Tn-i-i given {To,Yo,... ,Tn,Yn) = Zn, and let A : D{E) D{E) denote 
the total compensator of Z. From the explicit representation of the compensator of 
counting processes given in Section 4.3 of we obtain that the hazard measure 
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of the conditional distribution is concentrated on (t„,oo) and given by 


(A.16) 



see (4.47) of [12], for paths z such that only n jumps are made on [0,t) and 
corresponding to having jump times ti,... ,tn with destination states yi,..., 

As the hazard measure is absolutely continuous, we obtain that the hazard function 
for the distribution exists and is given by t i-A t — tn), the Radon- 

Nikodym derivative of the hazard measure. This proves the result. □ 


Lemma A.3. The conditional distribution of Yn+i given To,Yo,... ,Yn,Tn+i is 
almost surely given by 


(A.17) 


p(y„ = j|To,ro,...,r„,T„+i) 


^n+l) 
Tn+l) ’ 


understanding that on the almost sure set when the above holds, the denominator 
is nonzero. 


Proof of \A.3[ With V{E) denoting the power set of E, let : 'P{E) —>■ [0,1] de¬ 
note the conditional distribution of Yn+i given (Tq, Fq; ■ • ■; U, Yn, = (z„, t). 

Let A : D{E) —>• D{E) denote the total compensator of F, and let the compensator 
for the jumps to state i be denoted by A* : D{E) D{E). By (4.48) of [H], we 
have that whenever qy^{t, t — tn) 5 ^ 0, it holds for all A C A that 


(A.18) 




ieA 


ieA 


qVn (^; I In) 


which yields the result. 


□ 


Lemma A.4. The conditional distribution of T„+i given Zt = i, Ut = u and 
Nt = n has hazard function h : ]R_|_ —>■ R+ given by h(s) = l(s>t)<J'i(L u + s — t). 


Proof of Lemma [X^] Let s > 0. We find that 

P{Tn+i < s\Zt =i,Ut= u, Nt = n) 

= P(Tn+l < s\Yn = i,Tn=t-U, T^+l > t) 

(A 191 - ^ ^1^" = i,Tn = t - u) 

P{Tn+l > t\Yn = i,Tn = t - u) 

Now, by Lemma IA.21 the distribution of T^+i given F„ = i and Tn = t — u has 
hazard function v 1 —5> l(v>t-u)qi(,v,v — ft — u)). Therefore, we in particular obtain 


(A.20) P(Tn+i > tlYn = i,Tn = t - u) = exp 


qi(v,v - ft- u)) dn 
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and, for s > t, 


(A.21) 


ds 


P{Tn+l < S,Tn+l > t\Yn = i,Tn = t - u) 


= qi{s, s - {t - u)) exp qi{v,v - {t - u)) dn^ . 


Combining (|A.19|1 . (jA.20|l and (IA.21I1 yields that for s > t, it holds that 


P{Tn+i < s\Zt = i,Ut = u, Nt = n) 

qi{v,v - {t- u)) di; 

proving that the conditional distribution of Tn+i given Zt = i, Ut = u and Nt = n 
has hazard s i—>■ qi{t,u + s — t) for s > t. As the support of the distribution is 
contained in [t, oo), this proves the lemma. □ 

Lemma A.5. The conditional distribution ofT{t) given Zt = i and Ut = u has 
hazard function h : ]R_|_ —>■ ]R_|_ given by h{s) = l(s>t)9i(i, u + s — t). 


(A.22) 


= 9 j(s, s - {t - u)) exp 



Proof of Lemma [A . 51 Define g : Nq ^ [0,1] by g{n) = P{Nt = n\Zt = i,Ut = u). 
Applying Lemma IA.4I it then holds for s > t that 


P{T{t) < s\Zt=i,Ut=u) 


(A.23) 


P{T{t) > s\Zt = i,Ut = u, Nt = n)g(n) 

n—0 
oo 

P{Tn+i > s\Zt =i,Ut= u, Nt = n)g{n) 

n—{) 


y^expf/ qi{v,u + v -t)dv] g{n) 

n=o / 

exp qi{v,u + V — t) dv'^ , 


yielding the result. □ 

From a heuristic perspective, Lemma IA.5I allows us some insight into the distri¬ 
bution of Z. The lemma states that given the values of {Zt,Ut), the distribution 
next jump has a fixed hazard, independently of the number of jumps made. We 
are now ready to prove the results stated in Section [3] 
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Proof of Lemma HlTl For n > 0, define g{n) = P{Nt = n,Zt = i,Ut = u). We 
then have 

P{Nt+h-Nt>2\Zt=i,Ut = u) 

oo 

= ^ P{Nt+h -Nt>2\Zt = t,Ut = u,Nt = n)gin) 

n —0 

oo 

(A.24) = ^ P(t < Tn+i < Tn +2 <t + h\Zt =i,Ut = u, Nt = n)g{n). 

n—0 

Now, by Lemma IA.4[ the conditional distribution of T^+i given Zt = i, Ut = u 
and Nt = n has hazard function qi(s,u + s — t) for s > t. Applying Lemma |A.2[ 
we then obtain that the conditional distribution of and Tn+i given Zt = i, 

Ut = u and Nt = n has density 

qi{v,u + V — t) du 

for s > t and zero otherwise. Next, noting that we always have t/r„+i = 0, we 
may use the strong Markov property of (Z, U) at T„+i and obtain 

P(t < Tn+i < r „+2 <t + h\Zt =i,Ut= u, Nt = n) 

(A.26) = X! / F’(s < r „+2 < t + /i|^T„+i = fc,T'n+i = s)/(fc,s)ds. 

keE''* 


(A.25) f{k,s) = q^k{s,u + s-t)exp^- 


Here, the distribution of T „+2 given = k and r„+i = s has hazard qkiv, v—s) 

for V > s and zero otherwise, which yields the conditional density 

(A.27) f{v\k,s)=qkiv,v-s)exp^-J^ qk{r,r - s) dr'^ . 

for V > s and zero otherwise. We thus obtain 

P{s < Tn+2 <t + h\ZT„+t = k, Tn +1 = s)= fiv\k, s) du 

J s 


(A.28) 

Inserting this in (IA.26p . we obtain the bound 




< 


||Q(u,u - s)||oo dv. 


P{t < r„+i < r „+2 <t + h\Zt = i,Ut = M, Nt = n) 

/ t-\-h pt-\-h 

/ - s)||oo||Q(s, u + s - t)||ooduds. 

As this bound is independent of n, we may use it in (|A.24p and obtain the required 
result. □ 
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Proof of Lemma \3.S\ By Lemma 13.11 we have 

\p{Nt+h-Nt>2\Zt=i,Ut=u) 

a 

(A.30) <h sup ||Q(s,m + s - - s)||oo- 

t<s,v<t-\-h 

Now, by the bound (13.6p . we find that [s,v) i-A ||(5(s,i^)||oo is bounded in both a 
neighborhood of (t, u) and (t, 0). Therefore, the above tends to zero as h tends to 
zero. □ 


Proof of Theorem \3.3\ . We will show the result with e = 1. To obtain (13.91) 
for this case, we first fix t, u > 0 and i G E and seek to bound the sum of 
t + h,u) — lij) over j ^ i for h < 1. Fix i j, we then have 

p^j(t,t + h,u) < P{Nt+h - Nt = l,Zt+h = j\Zt =i,Ut= u) 

(A.31) + PiNt+h -Nt>2, Zt+h = j\Zt = i,Ut = u). 

Next, note that by Lemma FA.31 and Lemma FA.51 we have 

P{Nt+h - Nt = l, Zt+h = j\Zt = i,Ut = u) 

qij{s,u + s — t) exp qi{v,u + V — t) dv^ ds 

nt-\-h 


< 


(A.32) 


< 


/ t+n 

qij{s, u + s — t) ds. 


since Nt+h — Nt = 1 and Zt+h = j is equivalent to having the next jump strictly 
after t occurring in the interval (t, t+h] with destination state j, and the jump after 
that occurring strictly after t + h. Applying this bound in (IA.31|) . the monotone 
convergence theorem yields 

''^^Pij{t,t + h,u) < / qi{s,u + s — t) ds 

(A.33) +P{Nt+h-Nt>2\Zt = i,Ut = u). 

Here, by Lemma l3.11 we have 

P{Nt+h-Nt>2\Zt = i,Ut = u) 


(A.34) 




< 




jIIQFu,-!; - s)||ooduds. 


Combining (|A.32p and (|A.34ll with (jA.3ip . we obtain for h < 1 that 
'^^Pij{t,t + h,u) 


< 


^ rt-hh / rt+h \ 

-J ||Q(s, u + s-t)||oo [ 1 + y ||(3(w,w-s)||ooduj ds 


(A.35) < sup ||(3(s,u + s —: 


1+ sup ||(3(i;,u-s) 


ds. 
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As we also have 


(A.36) 


+ h,u)-l 




we finally obtain 



(A.37) <2 sup ||Q(s,m + s - t)||oo ( 1 + sup ||(5(u,u 



ds. 


Letting C(t,u) be the right-hand side of (IA.35I) . we obtain (13.91) for the case 
i 7 ^ j. By the right-continuity of the intensities, this definition of the mapping 
is measurable, and by dsm, it is bounded on compacts. 


Proof of Theorem \3.4\ Consider first the case i ^ j. Here, we need to show that 

(A.38) qij{t,u) = lim + h,u), 

h^o n, 

To this end, we first make the decomposition 


p^j{t,t + h,u) = P{Nt+h - Nt = I, Zt+h = j\Zt =i,Ut = u) 
(A.39) + PiNt+h -Nt>2, Zt+h =j\Zt = i,Ut = u). 


Here, Lemma [3.21 shows that the second term tends to zero, so it suffices to show 
that the hrst term tends to qij{t, u). To do so, we note that having Nt+h — Nt = 1 
and Zt+h = j is equivalent to having T{t) <t + h, T{T{f)) > t-\-h and ^T(t) = j- 
Defining 


(A.40) 

(A.41) 


/(t,u, s) = qij{s,u + s -t) exp qi{v,u + v - t) du^ 

a{t,r) =exp(^-qj{v,v)dv^, 


for s > t and r > t, we find by Lemma |A.5I and Lemma |A.3I that s i— f{t, u, s) is 
the conditional density given Zt = i and Ut = u oi making the next jump at time 
t to state j, and r i—a(t, r) is the survival function for the next jump given Zt = j 
and Ut = 0. Therefore, we obtain 


(A.42) 


= P{Nt+h - Nt = l, Zt+h = j\Zt = i,Ut = u) 



fit, u, s)a(s, t + h) ds. 


As the intensities are assumed to be right-continuous, this yields 
(A.43) lim ^P(Nt+h - Nt = l,Zt+h = j\Zt = i,Ut = u) = qij{t,u), 

h^O tl 


□ Q 
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which, combined with (|A.39I) . yields (|A.38p . It remains to prove for any i £ E 
that 

(A.44) q^^(t,u)= \im^{pii{t,t + h,u)-l). 

h^O tl 

In order to obtain this, we simply apply the dominated convergence theorem with 
the bound C{t,u) from Theorem 13.31 to obtain 

lim y{pu{t,t + h,u) - 1) = - lim W yPij{t,t + h,u) 
ft-s-o n h^o ^' h 

= - lim ^pij {t, t + h,u) 

(A.45) == qu{t,u), 

jA* 

as required. □ 


A.3. Proofs for Section|^ We begin by proving a few auxiliary lemmas, includ¬ 
ing a version of the Chapman-Kolmogorov equations for semi-Markov processes. 

Lemma A. 6. It holds for all 0 < s < t that Ut — Us < t — s. 


Proof of Lemma lA.dl Fix 0 < s < t. If there are no event times in (s,t], then 
Lemma FA . 1 1 shows that Ut — Us = t — s. Considering the case where there is an 
event time in (s, t], let T be the largest such event time, T = sup{T„|s <Tn < t}. 
We then obtain 

(A.46) Ut-Us=t-T-Us<t-T<t-s, 

as required, since U is nonnegative. □ 

Lemma A. 7 . Let 0 < s < t, and let u > 0. For i ^ j , it holds that the measure 
Pij{s,t,u-i') is concentrated on the set [0U~s\- ForiGE, it holds that the measure 
Pii{s, t, u, •) is concentrated on [0, t — s] U {u, u + t — s}. 


Proof of Lemma |A. ?t Consider first the case where i ^ j. For s = t, it is 
immediate that Pij{s, t^u,-) is the zero measure, so the result is immediate in this 
case. Consider instead the case where s < t. Here, we have 


Pij(s,t,u,A) = P{Zt =j,Ut e A\Zs = i,Us =u) 

= P{Zs =i,Zt= j, Ut e A\Zs =i,Us= u). 


(A.47) 
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Now, when Zg = i and Zt = j, with s < t, it must hold that there is some event 
time for Z in the interval (s,t]. Therefore, we may apply Lemma lA.61 and obtain 


{Zg =i,Zt= j) C <Tn <t,Zg = i, Zt = j) 


C <Tn <t, Ut„ =Q,Zg = i, Zt = j) 

C < Tn < t,Ut < t — T„, {7t„ = 0,Zg = i, Zt = j) 


(A.48) 


C{Ut<t- s). 


Using this in (IA.47I) . we obtain 


p^j{s,t,u,A) = P{Zt = j,Ut e A\Zg =i,Ug = u) 

= P{Zs = i,Zt = j, C/t e An [0,t- s]\Zs = i,Ug = u), 


(A.49) 


as required. Next, consider a single i G E. In the case where s = t, it is immediate 
that pii{s, t, u, •) is concentrated on {m}, and so the result follows. Consider instead 
s < t. In this case, it either holds that there are zero event times in (s, f], or there 
is an event time. In the latter case, we can use a calculation as in (IA.48P to obtain 
that the duration at time t must be in [0, u + t — s]. In the former case, we have 


<^n=l {Tn ^ {s,t],Zs =i,Ug = u,Zt = j) 



□ 


The analogue of the Chapman-Kolmogorov equations for semi-Markov processes is 


encapsulated in the following lemma. Note that the integration domain in (IA.5ip 


is assumed to be [0, u-|-t — s] and not [0, M-|-t — s). This is important as Pij{s, r, it, •) 
in the case i = j can have a point mass in u + t — s, corresponding to the case 
where no jumps are made in the interval (s, r]. 

Lemma A.8. Fix 0 < s < r < t. It holds that 


(A.51) 

In particular, we have for d > 0 that 



(A.52) 



Proof of Lemma I A. A By the Chapman-Kolmogorov equations for the inhomoge¬ 
neous Markov chain {Z,U), we have 

Pr,t(fc,T, {j} X A)dPs^r{i,u, dk, dn) 



/ Pr,t{k,v,{j} X A)dPg^r{i,u,k, dv) 


(A.53) 
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As Lemma FA. 71 shows that the support of pik{s, r, u, ■) is included in [0, u + r — s], 
we obtain the result. The identity (|A.52|) follows from (|A.51(1 by definition. □ 

Proof of Lemma EH We first note that for Z to remain in state i at time t + h 
conditionally on being in state i at time t can only happen by changing state an 
even number of times. In particular, this yields 

(A.54) 

Pii{t, t + h,u,{u + h)-) = P{Zt+h = i, Ut+h <u + h\Zt = i,Ut = u) 

< P(Nt+h ~ = 0, Ut+h < u + h\Zt = i,Ut = u) 

+ PiNt+h -Nt> 2\Zt =i,Ut = u). 


Here, we have 


P{Nt+h — Nt — 0, Ut+h < u + h\Zt — i,Ut — u) 

(A.55) = P{Nt+h -Nt = Q,Ut = u, Ut+h <u + h\Zt = i,Ut = u) = 0, 

since, when making no jumps in the time interval (t,t + /i], Ut = u implies that 
Ut+h = u + h. As a consequence, we obtain 

(A.56) ^Puit,t + h,u, {u + h)-) < ^P{Nt+h - Nt> 2\Zt = i,Ut = u). 
n n 

The limit statement (IQ) then follows from Lemma and the existence of the 
bound (14.311 follows as in the proof of Theorem 13.31 □ 


In order to prove Lemma 14.21 we first define 

(A.57) rij{s,u,t) = qijit,u + t — s) exp J qi{v,u + v — s) dv^ , 

(A.58) i?i(s, u, t) = exp y gi(u, u + u — s) du^ 


Note that {j,t) Tij{s,u,t) is then the conditional density of the next jump and 
its destination state given Zg = i and Ug = u, and t i—i?i(s,it,t) is the survival 
function for the next jump given Zg = i and Ug = u. 


Proof of Lemma \4-‘d\ First consider the case where i ^ j. Note that as Pij{s, t, u, •) 
is concentrated on [0, t — s] by Lemma I A. 71 we have that whenever d — h >t — s, 
it holds that 


(A.59) Pii{s, t, u, d) — Pij{s, t,u,d — h) = p{s, t, u, (d — h, d]) — 0. 

In particular, if d > t — s, it is immediate that the derivative from the left exists 
and is zero. Next, consider the case where d < t — s. Ass<t — din this case, we 
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can apply the Chapman-Kolmogorov equations to obtain 
lim - pij{s,t,u,d— h) 

tl 

= lim -rp{s, t, u, {d — h, d]) 
h —^0 h 

^ nU-\-t — d — S 

(A.60) = X! r / Pkj{t-d,t,v,{d-h,d])pik{s,t-d,u,dv), 

kGE "'U 

insofar as the limits exist. We wish to argue that for each k in the sum above, the 
limit exists. To this end, we first consider the case k = j. Note that if Ut-d = v 
for u > 0 and there are no jumps on (t — d, t], then Ut > d. Therefore, we obtain 

Pjj{t — d, t, V, {d — h, d]) 

= P{Zt = j,d- h <Ut < d\Zt-d = j, Ut-d = v) 

(A.61) = PiNt - Nt-d >2,Zt=j,d-h<Ut< d\Zt-d = j, Ut-d = v). 

Here, Lemma [3.21 yields 

lim sup j-Pi^t - Nt-d >2,Zt= j,d-h<Ut< d\Zt-d = j, Ut-d = v) 
/t-s-o n. 

(A.62) < limsup j;P{Nt - Nt-d > 2\Zt-d = j, Ut-d =v) =0. 

h^o n 

Therefore, applying Lemma 13.11 and the dominated convergence theorem, we ob¬ 
tain 

nU-\-t — d — S 

(A.63) 1™ T / Pjj{t — d,t,v,{d — h,d\)pij{s,t — d,u, dv) = 0. 

h-to h Jq 

Next, consider the case k ^ j. In this case. Lemma lATfl yields that pkj{t — d, t, v, •) 
is concentrated on [0, d]. Also note that when Ut > d — h, no jumps are made in 
the time interval (t — d + h,t]. Therefore, any jumps made by the process in the 
interval {t — d, t] must in this case be made in the interval [t — d^t — d + h], yielding 

Pkj{t - d,t,v, (d- h,d]) 

= P{Zt = j,d - h < Ut\Zt-d = k, Ut-d = v) 

= P{Nt-d+h - Nt-d = l,Zt=j,d-h< Ut\Zt-d = k,Ut-d = v) 

+ P{Nt-d+h — Nt-d P 2, Zt = j,d — h < Ut\Zt-d = k, Ut-d = v). 

As in the previous case. Lemma 13.21 yields 

limsup - Nt-d >2,Zt=j,d-h< Ut\Zt-d = k,Ut-d = v) 

/ i ->0 “ 

(A.64) < limsup ^P{Nt-d+h - Nt-d > 2\Zt-d = k, Ut-d = v) = 0, 

h^o n 

with the convergence, as before, being bounded above by a constant according 
to Lemma 13.11 Next, note that on Nt-d = n, having Nt-d+k — Nt-d = 1 and 
d — h < Ut is equal to having t — d < T„+i < t — d + h and Tn +2 > t. Let 
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g{n) = P{Nt-d = n\Zt-d = k, Ut-d = v) and A = {t — d,t — d + h], we therefore 
obtain 


P{Nt_d+h - Nt-d = l,Zt= j,d-h< Ut\Zt-d = k, Ut-d = v) 

OO 

(A.65) = ^ P{Tn+i e A, Tn+2 > t,Zt= j\Zt-d = k, Ut-d = v, Nt-d = n)g{n). 

n—0 

Here, by Lemma lA.dl and Lemma lA.Si the conditional distribution of 
given Zt-d = k Ut-d = v and Nt-d = n has density {l,s) i-A rki{t — d,v,s) for 
I 7 ^ k and s > t — d, and by Lemma IA.2[ the survival function of the conditional 
distribution of Tn+2 given Tn+i = s and = I is w 0, iu ). Here, we 

use the notation outlined in (IA.57I) and (IA.58I) . Thus, we obtain 

P(r„+i e A, Tn +2 >t,Zt= j\Zt-d = k, Ut-d = V, Nt-d = n) 

pt — d-\-h 

(A.66) = / rkj{t — d,v, s)Rj{s,0,t) ds. 

J t—d 

As this is independent of n, insertion in (IA.65I) yields 

P{Nt-d+h - Nt-d = ^,Zt = j,d- h < Ut\Zt-d = k,Ut-d = v) 

pt — d-\-h 

(A.67) = / — d,v, s) ds. 

J t—d 

By right-continuity of the intensities, this yields 

lim ^P{Nt-d+h - Nt-d = l,Zt = j,d - h < Ut\Zt-d = k,Ut-d = v) 
/i->o n 

= fkj {t — d,v,t — d)Rj {t — d, 0, t) 

(A.68) =qkj{t-d,v)ejip(- [ qj{r,r - (t - d)) dr 

V Jt-d 


For brevity, we let akj{t,d,u,v) denote the right-hand side of (IA.68I) . Now using 
(IA.63I) and (IA.68I) in (IA.60I) . we obtain 


(A.69) 


lim —pij{s, t, u, d) — Pij(s, t,u,d — h) 
h—>-0 h 

^ pu-\-t—d—s 

= lim - / pkj{t - d,t,v,{d- h,d\)pik{s,t 



akj{t,d,u,v)p^k{s,t 


d, u, du), 


d, u, du) 


yielding (14.41) . Note that in order to move the limit under the integral in (IA.69I) . 
we applied the dominated convergence theorem, making use of that the integral in 
(IA.67I) is bounded from above on \t — d,t — d + e] for some e > 0. This shows that 
Pij(s, t, u, d) is differentiable from the left in d for d > 0 when i ^ j. The diagonal 
case follows immediately from this by writing pii{sU,u,d) as one minus the sum 
oi pij{s,t,u,d) foT j ^ i. □ 
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Proof of Theorem \4.3\ Let s,u > 0 and let t > s and d > 0. Applying the 
Chapman-Kolmogorov equations of Lemma IA.8I we have 

= lim {s,t + h,u, d) -pij{s,t,u, d)) 

h^O il 

(A.70) = 1™ r X] / pkj{t,t + h,v,d)ptk{s,t,u, dv) --pij{s,t,u,d), 

h — ^0 hj fr\ iL 


insofar as the limits exist, our objective is to argue that this is the case. Now, 
for k ^ j, Lemma I A. 71 shows that the measure pkj{t,t + h,v,-) is concentrated 
on [0, h]. Therefore, whenever d > h, we have Pkjit, t + h,v,d) = Pkjif, t + h,v). 
Applying Lemma 13.41 Lemma 13.31 and the dominated convergence theorem, we 
then obtain 


(A.71) 




lim 

h—^0 


h 


Pkjit,t + h,v,d)pik{s,t,u, du) 


lim -Pkj{t,t + h, v)p^k {s,t,u, dv) 
■) h —>^0 ri 


nu-\-t — s 


qkj{t,v)pik{s,t,u, dw). 


Now consider the case k = j. We wish to evaluate the limit 


j-u+t — s 2 

(A.72) lim/ -pjj{t,t + h,v,d)pij{s,t,u, dv) --pij{s,t,u,d). 

h^O Jo II II 


We note that for v > d, Lemma nA\ yields 


(A.73) 


lim sup -rPjjit, t + h,v,d) < lim sup -rPjj{t, t + h,v,{v + h)—) = 0, 
h —^0 h h —>-0 h 


so that pjj (t,t + h,v,d)/h tends to zero as h tends to zero from above. The bound 
in Lemma 14.11 and the dominated convergence theorem then allows us to conclude 
that in the case d < u + t — s, we have 

nU-\-t — S ^ 

(A.74) lim / —pjj{t,t + h,v,d)pij{s,t,u,dv)=0. 

'*^0 Jd h 

Therefore, we obtain that if d < m + t — s, then the limit (IA.72I) is equal to the 
limit 

I 1 

(A.75) lim / -pjj{t,t + h,v,d)pij{s,t,u, dv) --pij{s,t,u,d), 

ii-s-o Jo h n 

provided that this exists. In the case d > u +1 — s, we we note that according to 
Lemma IATtI pij{s, t, u, ■) is concentrated on [0,u + t — s], so the limits (IA.72I1 and 
(IA.75I1 are also equal in this case, whenever they exist. 


We proceed with evaluating (jA.75|) . Note that Pjj{t,t + h,v,-) is concentrated 
on [0,i; + h] by Lemma I A. 71 Therefore, pjj{t,t + h,v,d) = pjj{t,t + h,v) for 
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0 < V < d — h, for d — h < V < d, we have 


Pjj (f, t + h,v,d) = Pjj(t, t + h,v,v + h) — Pjj{t, t + h,v, {d, v + h]) 
(A.76) = Pjj{t,t + h,v) — Pjj{t,t + h,v, {d,v + h]). 


As a consequence, we find that 


(A.77) 


f‘^ 1 1 

/ J^Pjj {t,t + h,v, d)pij (s, t, u, du) - -p,j {s, t, u, d) 

1 1 

= J J^Pjj (^. i '^)P^J {s,t,u,dv)- -pij {s,t,u,d) 

1 

~T Pjj{t,t + h,v,{d,v + h])ptj{s,t,u, dv), 

" Jd-h 


where the final integral is over {d — h^d]. Taking the limit of the first two terms in 
(TOtI) . we may apply the dominated convergence theorem as in the off-diagonal 
case and obtain 


(A.78) 


1 1 

lim / -pjj {t, t + h, v)pij (s, t, u, du) - -pij (s, t, u, d) 
h—^o Jq n a 

= lim / + h,v) - l)pij{s,t,u, dv) 

h^OjQ ti 

= / qjj(t,v)p^jis,t,u, dv). 

Jo 


As regards the term being subtracted in (IA.77I1 . we can write 


1 

- / pjj{t,t + h,v,{d,v + h])pij{s,t,u, dv) 

P Jd-h 

= ^{Pij{s,t,u,d) - pij{s,t,u,d- h)) 

1 

(A.79) +7:/ iPjj(t,t + h,v,{d,v + h]) - l)pij{s,t,u, dv). 

P Jd-h 


For the first term in (|A.79|) , we may use Lemma 14.21 to obtain that the limit as h 
tends to zero exists and is given by 
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where the right-hand side denotes the left derivative. As for the second term in 
(fAT^ . we have 


(A.81) 


1 f 

- / + h,v,{d,v + h]) - l)pij{s,t,u, dv) 

“ Jd-h 

1 1 

/ T (Pn {t,t + h,v) - 1) + -rPjj {t,t + h,v, d)pij (s, t, u, di;) 

Jd-h “ " 

rd ^ 

/ T iPjj t + h,v) - l)pij (s, t, u, dn) 

Jd-h “ 

fd ^ 

/ TPjj {t,t + h,v, d)pij {s, t, u, di;) 

Jd-h « 


Here, the integrand in the first term of the final estimate converges to qjj(t,v), 
and by Theorem 13.31 we also have ioi d — h < v < d that 

(A.82) \{pjj{td d h,v) - 1) < sup C{t,u), 

“ d-h<v<d 


the latter being finite. Therefore, the dominated convergence theorem yields that 
the first term converges to zero. As regards the second integrand, we note that 
d — h < V < d implies d < v + h, so Lemma [4.11 yields that the integrand in this 
case tends to zero, dominated by a bound measurable in t and u and bounded 
on compacts. Therefore, the dominated convergence theorem also allows us to 
conclude that the second integral also tends to zero. All in all, this yields 


(A.83) 


lim 

h^O 



■j^{Pjjit,t + h,v, {d,v + h]) - l)pij{s,t, u, du) 


= 0 . 


Combining (IA.83I) with our previous conclusions and simplifications, we finally 
conclude 

pu+t—s 2 2 

lim / -pjj{t,t + h,v,d)pij{s,t,u, dv) --pij{s,t,u,d) 

h^ojQ h n 

(A.84) =J qjj{t,v)p^j{s,t,u, dv)-^^{s,t,u,d), 

and from (jA.7111 and (jA.84p . the theorem follows. □ 
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